Information processing apparatus, information processing method, and program

ABSTRACT

Provided is an information processing apparatus including a detection unit that detects a difference from a previously detected marker in a predetermined marker, by image recognition processing on an image obtained by image-capturing an arrangement place of an article; and an execution unit that executes a process in accordance with the difference.

TECHNICAL FIELD

The present invention relates to an information processing apparatus, aninformation processing method, and a program.

BACKGROUND ART

An example of an apparatus that surveils a situation of commodities thatis placed on display is disclosed in Patent Document 1. Based on imagedata of an image capture camera, a surveillance apparatus in PatentDocument 1 detects whether or not a commodity is moved, and detects adirection of a person that stays in the vicinity of an area where thecommodities are arranged, and based on a result of the detection,outputs warning information. For example, in a case where it is detectedthat a commodity disappears from a commodity arrangement area and whereit is detected that a customer is not facing a direction of thecommodity arrangement area, a message notifying that there is alikelihood that the commodity has disappeared due to a wrongful act suchas shoplifting is output.

Furthermore, In Patent Document 2, an inventory management apparatus isdisclosed that checks for stock-out. The inventory management apparatusthat is disclosed in Patent Document 2 includes a mark that is printedin ink with high reflectivity with respect to light in an infraredregion, an object to be managed that is positioned on the mark, animage-capturing unit that image-captures the mark, an image processingunit that identifies the mark from a captured image, and a centralprocessor which determines, in a case where the mark is not recognized,that the object to be managed is present and a stock is available, andwhich determines, in a case where the mark is recognized, that theobject to be managed is not present and acquires information on theobject to be managed that is not present from the identified mark. Withthis configuration, the stock-out can be checked for in an easiermanner.

RELATED DOCUMENT Patent Document

[Patent Document 1] Japanese Patent Application Publication No.2015-176227

[Patent Document 2] Japanese Patent Application Publication No.11-281754

SUMMARY OF THE INVENTION Technical Problem

In the technologies in the patent documents, which are described above,the disappearance of the commodity is detected, the direction of theperson who stays in the vicinity of the commodity that disappears isdetected, the detection of a wrongful act such as shoplifting or runningout of the commodity is detected, and the commodity that runs our isidentified. Thus, the inventory management can be performed. However, aprocessing load on the information processing apparatus increases.

An object of the present invention, which was made in view of thesituations described above, is to provide an information processingapparatus, an information processing method, and a program that detectan increase and a decrease in the number of commodities in a simple andefficient manner.

Solution to Problem

In aspects of the present invention, the following configurations areemployed in order to solve the problems described above.

A first aspect relates to an information processing apparatus.

According to the first aspect, there is provided an informationprocessing apparatus including: a detection unit that detects adifference from a previously detected marker in a predetermined marker,by image recognition processing on an image obtained by image-capturingan arrangement place of an article; and an execution unit that executesa process in accordance with the difference.

A second aspect relates to an information processing method that isexecuted by at least one computer.

According to the second aspect, there is provided an informationprocessing method executed by an information processing apparatus, themethod including: detecting a difference from a previously detectedmarker in a predetermined marker, by image recognition processing on animage obtained by image-capturing an arrangement place of an article;and executing a process in accordance with the difference.

Note that according to another aspect of the present invention, theremay be provided a program that causes at least one computer to performthe method according to the second aspect and there may be provided acomputer-readable storage medium on which the program is stored. Thestorage media include a non-transitory tangible medium.

The computer programs include a computer program code that, when causedby the computer to be executed, causes the computer to perform theinformation processing method on the information processing apparatus.

Note that results which are obtained by converting any combination ofthe constituent elements described above and an expression in thepresent invention in the method, the apparatus, the system, the storagemedium, the computer program, and the like is also effective as aspectsof the present invention.

Furthermore, various constituent elements do not necessarily need to bepresent independently of each other. A plurality of constituent elementsmay be formed as one member, one constituent element may be formed as aplurality of members, a certain constituent element may be a part of anyother constituent element, a part of a constituent element and a part ofany other constituent element may be present redundantly, and so forth.

Furthermore, a plurality of procedures are described in sequential orderin the method and the computer program according to the presentinvention, but the sequential order of the descriptions does not limitan order in which a plurality of procedures are executed. Therefore,when the method and the computer program according to the presentinvention are implemented, the order of a plurality of procedures can bechanged within a permissible range in contents.

Moreover, a plurality of procedures in the method and the computerprogram according to the present invention are not limited to beingindividually executed at different timings. Therefore, when any otherprocedure is executed while a certain procedure is executed, a timing atwhich the certain procedure is executed and a timing at which the anyother procedure is executed may overlap partially, and so forth.

Advantageous Effects of Invention

According to each aspect described above, an information processingapparatus, an information processing method, and a program that detectan increase and a decrease in the number of articles of commerce in asimple and efficient manner.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects described above, and other objects, features and advantagesare further made apparent by suitable example embodiments that will bedescribed below and the following accompanying drawings.

FIG. 1 is a functional block diagram logically illustrating aconfiguration of an information processing apparatus according to anexample embodiment of the present invention.

FIG. 2 is a diagram schematically illustrating a system configuration ofan information processing system which includes the informationprocessing apparatus according to the example embodiment of the presentinvention.

FIG. 3 illustrates an installation example of a marker according to thepresent example embodiment.

FIG. 4 is a diagram illustrating an example of a configuration of acomputer according to the example embodiment of the present invention.

FIG. 5 is a diagram illustrating an example of a data structure of astorage unit, which relates to each piece of information that is storedin a storage apparatus that is accessed by the information processingapparatus according to the present example embodiment.

FIG. 6 is a diagram illustrating an example of a data structure of adetection result storage unit of the storage apparatus that is accessedby the information processing apparatus according to the present exampleembodiment.

FIG. 7 is a diagram illustrating an example of a condition forprocessing by the information processing apparatus according to thepresent example embodiment.

FIG. 8 is a flowchart illustrating an example of operation of theinformation processing apparatus according to the present exampleembodiment.

FIG. 9 is a diagram for describing marker detection processing by theinformation processing apparatus according to the present exampleembodiment.

FIG. 10 is a functional block diagram illustrating a logicalconfiguration of the information processing apparatus according to theexample embodiment of the present invention.

FIG. 11 is a diagram for describing the marker detection processing bythe information processing apparatus according to the present exampleembodiment.

FIG. 12 is a diagram illustrating an example of a condition forprocessing by the information processing apparatus according to thepresent example embodiment.

FIG. 13 is a diagram illustrating an example of marker installation forthe information processing apparatus according to the present exampleembodiment.

FIG. 14 is a diagram for describing the marker detection processing bythe information processing apparatus according to the present exampleembodiment.

FIG. 15 is a diagram illustrating an example of the condition for theprocessing by the information processing apparatus according to thepresent example embodiment.

FIG. 16 is a flowchart illustrating an example of operation of theinformation processing apparatus according to the present exampleembodiment.

FIG. 17 is a functional block diagram illustrating a logicalconfiguration of an information processing apparatus according to anexample embodiment of the present invention.

FIG. 18 is a diagram for describing the marker detection processing bythe information processing apparatus according to the present exampleembodiment.

FIG. 19 is a diagram illustrating an example of a data structure of adetection result storage unit of a storage apparatus that is accessed bythe information processing apparatus according to the present exampleembodiment.

FIG. 20 is a diagram illustrating an example of the condition for theprocessing by the information processing apparatus according to thepresent example embodiment.

FIG. 21 is a flowchart illustrating an example of the operation of theinformation processing apparatus according to the present exampleembodiment.

FIG. 22 is a diagram illustrating an example of the marker installationfor the information processing apparatus according to the presentexample embodiment.

FIG. 23 is a diagram conceptually illustrating an example of a systemconfiguration of an example of the information processing apparatusaccording to the present invention.

DESCRIPTION OF EMBODIMENTS

Example embodiments of the present invention will be described belowwith reference to the drawings. Moreover, in all the drawings, the sameconstituent elements are given the same reference numerals, anddescriptions thereof will not be repeated.

First Example Embodiment

An information processing apparatus according to a first exampleembodiment of the present invention, an information processing method,and a program will be described below.

FIG. 1 is a functional block diagram logically illustrating aconfiguration of an information processing apparatus 100 according to anexample embodiment of the present invention.

The information processing apparatus 100 includes a detection unit 102that detects a marker that is determined in advance, which is differentfrom a commodity, by image recognition processing on an image obtainedby image-capturing a place on which a commodity is placed, and anexecution unit 104 that executes a process that is determined inadvance, when a result of the detection of the marker satisfies acondition that is determined in advance.

The information processing apparatus 100 is further connected to astorage apparatus 110 in an accessible manner. The storage apparatus110, for example, is a hard disk, a Solid State Drive (SSD), a memorycard, or the like, and may be included in the information processingapparatus 100 and may be an external apparatus.

FIG. 2 is a diagram schematically illustrating a system configuration ofan information processing system 1 which includes the informationprocessing apparatus 100 according to the example embodiment of thepresent invention.

In each of the figures in the present specification, a part of aconfiguration that does not substantially relate to the gist of thepresent invention is not illustrated.

The information processing system 1 has an image capture camera 10, aserver computer 20, and a cash register 30.

The image capture camera 10 is one that image-captures a commodityshelving unit 40 in a shop, a stock storage warehouse, or the like. InFIG. 2, one image capture camera 10 is illustrated, but a plurality ofimage capture cameras 10 may be installed. The image capture camera 10image-captures commodities 42 that are placed on display or stored onthe commodity shelving unit 40, and a marker 50 that is installed in aposition that is determined in advance in the vicinity of a place wherethe commodities 42 are placed. A plurality of commodities 42 are assumedto be placed on the commodity shelving unit 40. For example, in a casewhere one or more commodities 42 remain on the commodity shelving unit40, at least a part of the marker 50 is hidden by the commodities 42 andthe at least a part of the marker 50 is also not present image data 12that is image-captured by the image capture camera 10.

The marker 50 is assumed to be one that is different from the commodity42. The marker 50, for example, can be expressed in various forms, suchas a logotype, a symbol mark, a symbol, a design, a pattern, aphotograph, a picture, a letter, a number, a scratch, and a barcode, andmay be expressed as a combination of two or more of these. Furthermore,the marker 50 may not be associated with the commodity 42, and may havea meaning and may not have a meaning. Furthermore, the marker 50 is notparticularly limited in terms of a size, a shape, a color scheme, thedegree of glossiness, the presence or absence of light emission, thenumber, and the like. Various forms are considered for the marker 50.Furthermore, the marker 50 may not have a planar structure, and may havea three-dimensional structure. Furthermore, in the example embodimentsthat will be described below, a configuration will be described in whichprocessing varies according to a type of the marker 50 or the number ofmarkers 50.

A method of installing the marker 50 is not particularly limited. Forexample, the marker 50 may be pasted on, be put on, be hung from, beattached to, be buried in, be printed on, or be drawing on a surface ofa shelf plate, a rear plate, or a prop of the commodity shelving unit40. Furthermore, an installation place of the marker 50 may be not onlythe commodity shelving unit 40, but also at least one of one piece offurniture, a wall, a ceiling, a floor, and a fixture in the vicinity ofthe place where the commodity 42 is placed.

Furthermore, the marker 50 may be installed on the commodity shelvingunit 40 or the like. In addition, at least a part of the commodityshelving unit 40 itself, or at least a part of any of one piece offurniture, a wall, a ceiling, a floor, and a fixture in the vicinity ofthe place where the commodity 42 is placed may be used as the marker 50.Alternatively, a display may be provided on a part of, or in thevicinity of (for example, on a rear surface, a shelf plate, or thelike), the commodity shelving unit 40, and the marker 50 may bedisplayed on the display.

Furthermore, in a specification in which the entire commodity shelvingunit 40 or each shelf plate is used as the marker 50, commodities thatare accommodated are different in size, shape, color, and the like foreach commodity shelving unit 40 or each shelf plate. Because of this,each commodity shelving unit 40 or each shelf plate in which thecommodities are accommodated can be used as the marker 50. For example,in an image area for image capture, which corresponds to each commodityshelving unit 40 or each shelf plate, in a case where a certain amountof difference is detected while time elapses, a predetermined message(an alert information relating to the occurrence of a large-scale theftor the like) may be transmitted, and a condition and details of aprocess may be set for every commodity shelving unit 40 or every shelfplate.

It is preferable that a relationship in arrangement among aninstallation position of the marker 50, an installation position(additionally, setting of an image capture direction, a viewing angle,and an image capture range is also included) of the image capture camera10, and an article (hereinafter also referred to as a “commodity”) 42 isas is described below.

The marker 50 and the image capture camera 10 are installed in such amanner as to establish a positional relationship such that the number,or an area of markers 50 present within an image capture range of theimage capture camera 10 changes between a case where the commodity 42 ispresent and a case where the commodity 42 disappears.

FIG. 3 illustrates an installation example of the marker 50 according tothe present example embodiment. For example, FIG. 3A illustrates a scenein which the marker 50 is pasted on a surface of a shelf plate 44 of thecommodity shelving unit 40. FIG. 3B illustrates a scene in which aplurality of commodities 42 are placed on the shelf plate 44 and inwhich the marker 50 is hidden by the commodity 42 from view. FIG. 3Cillustrates a scene in which almost all of the commodities 42 disappearand in which the marker 50 appears.

The image capture camera 10 is provided above the shelf plate 44 of thecommodity shelving unit 40. Moreover, the image capture direction, theviewing angle, and the image capture range of the image capture camera10 are set to allow image-capturing in which when the commodity 42 ispresent, an image of the marker 50 is hardly captured in the image data12 and when the commodity 42 disappears, the image of the marker 50 iscaptured into the image data 12.

With reference again to FIG. 2, the image data 12 obtained byimage-capturing with the image capture camera 10 is transmitted to theserver computer 20. A method of making a connection between andperforming communication between the image capture camera 10 and theserver computer 20 is not particularly limited. However, predeterminedciphering processing and an authentication procedure are assumed to beperformed, and thus, a secure communication environment is assumed to beprovided. The connection may be a wired one or a wireless one.

Furthermore, a direct connection or an indirection connection may bemade between the image capture camera 10 and the server computer 20.

For example, the image data 12 obtained by image-capturing with theimage capture camera 10 may be transmitted directly to the informationprocessing apparatus 100, and the information processing apparatus 100may sequentially receive the transmitted image data 12. Alternatively, astorage apparatus (for example, the storage apparatus 110) that isaccessible by the image capture camera 10 and the information processingapparatus 100 may be provided. The image data 12 obtained byimage-capturing with the image capture camera 10 may be stored in thestorage apparatus 110, and the information processing apparatus 100 mayread and acquire the stored image data 12 from the storage apparatus110.

In the present example embodiment, for example, a web camera, such as anInternet Protocol (IP) camera, is used as the image capture camera 10.The IP camera, for example, has a wireless Local Area Network (LAN)communication function, and may be connected to the server computer 20through a relay an apparatus (not illustrated) such as a router.

In the present example embodiment, a configuration is employed in whichthe image data 12 is output from the image capture camera 10, but nolimitation to this is imposed. The image data 12 may be image data on atleast one of a moving image and a still image. A data format is also notparticularly limited. Various formats can be employed in accordance withspecifications, standards, performances, and the like of the imagecapture camera 10 and the server computer 20.

In the present example embodiment, for example, the image capture camera10 uses formats that comply with Open Network Video Interface Forum(ONVIF). The image data 12, for example, may be stored in a file formatthat complies with Motion-Joint Photographic Experts Group (JPEG), orH.264 (Moving Picture Experts Group phase 4 (MPEG-4) Part 10 AdvancedVideo Coding (AVC)), in the storage apparatus 110.

The server computer 20, for example, includes a main body 22, a monitor24, and a keyboard 26. The server computer 20 is a computer thatrealizes the information processing apparatus 100 in FIG. 1, and anexample of a configuration of the main body 22, which will be describedbelow, is illustrated in FIG. 4.

The monitor 24 is a liquid crystal display or the like. Variousoperation screens of the information processing apparatus 100 accordingto the present example embodiment, result data 28 of the detection ofthe marker 50 by the detection unit 102 of the information processingapparatus 100, or the like are displayed on the monitor 24. Furthermore,the result data 28 of the detection of the marker 50 may be output tothe cash register 30 that will be described below.

The keyboard 26 is an example of an operation unit of the servercomputer 20. As the operation unit of the server computer 20, a mouse, atouch panel that is integrally combined with the monitor 24, or thelike, is considered in various ways.

The cash register 30, for example, has a shop-assistant display 32, acustomer display (not illustrated), a scanner (not illustrated), aninput button (not illustrated), a drawer (not illustrated), and a camera(not illustrated).

The shop-assistant display 32 is a display that has a function of atouch panel. Various screens are displayed on the shop-assistant display32. Along with this, the shop-assistant display 32 receives input thatis necessary for a shop assistant Ua to operate the cash register 30.The customer display is a display that has a function of a touch panel.Various screens are displayed on the customer display. Along with this,the customer display may receive input that is necessary for a customerto operate a screen.

The scanner is a handy-type scanner or a stationary-type scanner, and isvarious types of readers that are used for registering a commodity to bepaid for. The scanner is a bar-code reader that reads a code that isattached to a commodity, or a camera for image-recognizing the commodityor the code that is attached to the commodity. The input button is aninput unit to receive a pressing-down operation by the shop assistantUa. A coin or a paper currency is stored in the drawer. The camera mayimage-capture a face of a customer Ub who stays in the vicinity of thecash register 30, for example, the customer Ub who pays for a commodityin front of the cash register 30.

As described above, the cash register 30 may be connected to the servercomputer 20, and may receive the result data 28 of the detection fromthe server computer 20. Based on the received result data 28 of thedetection, the result of the detection of the marker 50 may be displayedon the shop-assistant display 32 of the cash register 30. A method ofmaking a connection between and performing communication between thecash register 30 and the server computer 20 is not particularly limited.However, predetermined ciphering processing and an authenticationprocedure are assumed to be performed, and thus, a secure communicationenvironment is assumed to be provided. The connection may be a wired oneor a wireless one. Furthermore, in FIG. 2, only one cash register 30 isillustrated, but a plurality of cash registers 30 may be provided.

In the present example embodiment, any one of the cash register 30 andthe server computer 20 (the information processing apparatus 100) isconfigured with a computer. FIG. 4 is a diagram illustrating an exampleof a configuration of this computer.

The computer 80 according to the present example embodiment includes aCentral Processing Unit (CPU) 82, a memory 84, a program 90 thatimplements constituent elements which are illustrated in FIG. 1, whichis loaded into the memory 84, a storage 85 in which the program 90 isstored, an Input/Output (I/O) 86, and a network connection interface (acommunication Interface (I/F) 87).

The CPU 82, the memory 84, the storage 85, the I/O 86, and thecommunication I/F 87 are connected to each other through a bus 89, andthe entire information processing apparatus 100 is controlled by the CPU82. However, a method of connecting the CPU 82 and the others to eachother is not limited to a bus connection.

The memory 84 is a memory, such as a Random Access Memory (RAM) or aRead Only Memory (ROM). The storage 85 is a storage device such as ahard disk, a Solid State Drive (SSD), or a memory card.

The storage 85 may be a memory such as a RAM or a ROM. The storage 85may be provided inside of the computer 80, and, if accessible by thecomputer 80, may be provided outside of the computer 80 and may beconnected to the computer 80 in a wired or wireless manner.Alternatively, the storage 85 may be removably provided to the computer80.

The CPU 82 loads the program 90, which is stored in the storage 85, intothe memory 84, for execution, and thus can implement a function of eachof the units of the information processing apparatus 100 in FIG. 1.

The I/O 86 performs input and output control of data and a controlsignal between the computer 80 and any other input and output devices.Examples of any other input and output device include input devices (notillustrated), such as the keyboard 26, a touch panel (not illustrated),a mouse (not illustrated), and a microphone (not illustrated), outputdevices (not illustrated), such as a display, a printer, and speaker,and an interface between one of these input and output devices and thecomputer 80, which are connected to the computer 80. Moreover, the I/O86 may perform control of the input and output of data into and fromother devices (not illustrated) which perform reading from and writingto a storage medium.

The communication I/F 87 is a network connection interface forperforming communication between the computer 80 and an externalapparatus. The communication I/F 87 may be a network interface forconnecting to a wired communication line, and may be a network interfacefor connection to a wireless communication line.

Each of the constituent elements of the information processing apparatus100 in FIG. 1 according to the present example embodiment is realized byany combination of a hardware piece of the computer 80 (the servercomputer 20) in FIG. 4 and a software piece. Then, it is apparent to aperson skilled in the art that various modifications to a method and anapparatus for such realization are present. A functional block diagramillustrating the information processing apparatus according to each ofthe example embodiments, which will be described below, illustrates ablock on a per-logical function basis instead of a configuration on aper-hardware basis.

The information processing apparatus 100 may be configured with aplurality of computers 80, and may be realized by a virtual server. Inthe present example embodiment, the information processing apparatus 100is configured with the server computer 20, but no limitation to this isimposed.

With reference again to FIG. 1, in the information processing apparatus100, the detection unit 102 detects the marker 50 by the imagerecognition processing of an image obtained by image-capturing the place(the commodity shelving unit 40) where the commodities 42 are placed. Asillustrated in FIG. 2, the information processing apparatus 100 (theserver computer 20) acquires the image data 12 from the image capturecamera 10.

In the present specification, the term “acquisition” means at least oneof an act (active acquisition) in which an apparatus itself fetches dataor information that is stored in any other apparatus or a storagemedium, for example, an act in which the apparatus itself makes arequest to any other apparatus for data or information to receive thedata or the information, an act in which the apparatus itself accessesany other apparatus or the storage medium to read data or information,or the like, and an act (a passive act) in which data or informationthat is output from any other apparatus is input into an apparatus, forexample, an act in which the apparatus receives data or information thatis delivered (transmitted, push-notified, or the like), or the like.Furthermore, the term “acquisition” also includes an act of selectingdata or information from among pieces of received data or information,for acquisition and an act of selecting data or information from amongpieces of delivered data or information, for reception.

FIG. 5 is a diagram illustrating an example of a data structure of astorage unit, which relates to each piece of information that is storedin the storage apparatus 110 that is accessed by the informationprocessing apparatus 100 according to the present example embodiment.

Identification information (a marker ID) on the marker 50, which isdetermined in advance, and feature information on the marker 50, arestored in the marker information storage unit 112 in FIG. 5A. Thefeature information on the marker 50, for example, may be image data onthe marker 50, and may be feature information that at least includes afeature point which is extracted from at least a part of the marker 50and a feature amount of the feature point. Furthermore, in a case wherethere is one type of markers 50 detected by the information processingapparatus 100, the identification information on the marker 50 may bedispensed with.

Moreover, information on a relative position to an installation area ofthe marker 50, a relative position to any other marker 50, or the likemay be included.

The feature point of the marker 50 and the feature amount of the featurepoint have the number of feature points that are necessary for thedetection unit 102 to detect the marker 50 from the image data 12, andthe feature amount of each feature point. Performance of the imagerecognition processing that is used by the detection unit 102, or thelike may be determined by a type of the marker 50.

The identification information on the marker 50 and information on aposition in which the marker 50 is installed are stored, in anassociated manner, in a marker positional-information storage unit 114in FIG. 5B. An example in which the marker ID is associated with thepositional information is illustrated in FIG. 5B, but the positionalinformation that is associated with every marker may be stored.

Positional information on the marker 50 is information that can identifya place where the marker 50 is installed. For example, at least one ofinformation indicating a store floor, information indicating an area ona floor, information indicating the commodity shelving unit 40, andinformation indicating a position of a shelf within the commodityshelving unit 40 or a position (which, for example, is expressed withcoordinates or a relative position to a reference) on the shelf isincluded. Alternatively, the positional information on the marker 50 maybe information that identifies the image capture camera 10 thatimage-captures the marker 50. Moreover, in a case where the imagecapture range of the image capture camera 10 is changeable, informationthat identifies the image capture range of the image capture camera 10that corresponds to the position in which the marker 50 is installed maybe included.

Furthermore, positional information that is acquired from a GlobalPositioning System (GPS), which corresponds to the position in which themarker 50 is installed is also not excluded.

For example, the detection unit 102 extracts a feature from an imagethat is captured by the image capture camera 10, by the imagerecognition processing, and compares a feature amount of the featurethat is extracted and a feature amount of the marker 50. In a case wherean area that corresponds to at least a part of the marker 50 has adegree of similarity that is equal to or more than a threshold, the areais detected as the maker 50.

The reason for using at least part of the area of the marker 50 insteadof the entire area, as will be described below, is because there is acase where the detection of the marker 50 is desired although a part ofthe marker 50 is hidden by the commodity 42. In a case where a part ofthe marker 50 is hidden by the commodity 42 and the entire area of themarker 50 is recognized, the marker 50 not being detected may be definedas a detection condition. In such a case, the entire area of the marker50 may be set as comparison targets.

A result of the detection of the marker 50 detected by the detectionunit 102 is stored in a detection result storage unit 120 in FIG. 6.

As illustrated in FIG. 6, detection date and time information is storedin the detection result storage unit 120 in a manner that is associatedwith a marker ID of the detected marker 50. In this example, aconfiguration is employed in which a detection date and time isassociated with every detected marker 50, but in another example,association of the marker ID of the marker 50 that is detected with adetection date and time may be avoided.

Moreover, feature information on an image area that corresponds to themarker 50 which is extracted from an image may be further stored in thedetection result storage unit 120 in association therewith.

The detection unit 102 may further information that is embedded inadvance into the marker 50, for example, various pieces of informationon the commodity 42 that are included in a QR code (a registeredtrademark), from the image area that corresponds to the marker 50. Forexample, various pieces of information on the commodity 42 may includeat least one of a commodity name, a brand name, a manufacturer name, astore name, a sales floor, a sales area, and the commodity shelving unit40.

The feature information on the image area, for example, may beinformation on the feature amount that is extracted, and includes atleast one piece of information indicating whether part or the whole ofthe marker 50 is detected and information indicating which part of themarker 50 is detected. Here, in a case where part of the marker 50 isdetected, a situation is assumed where the remaining part of the marker50 is not included in an image, for example, because a commodity 42remains and thus, the remaining part of the marker 50 is hidden. In acase where the whole of the marker 50 is detected, for example, asituation is assumed where almost all the commodities 42 disappear fromthe commodity shelving unit 40.

A method of determining whether part or the whole of the marker 50 isdetected is considered in various ways, and is described as follows.However, no limitation to the following is imposed. Furthermore, one ormore combinations of the followings may be made.

(a1) The feature amount of each feature point within the marker 50 isstored in advance in the feature information on the marker 50,inclusively in association with positional information. Then, frompositional information on a feature point that corresponds to thefeature amount that is detected, an area in which the detected featurepoints are included, within the marker 50, can be identified.

(a2) A size (a position) of an area that is occupied by the entiremarker 50, within a predetermined image capture range (determined byrelative positions of, or a distance between, the image capture camera10 and an image capture target, or the like) of the image capture camera10, is stored in advance in the feature information on the marker 50. Anarea of the detected area of the marker 50 is calculated and is comparedwith an area of the entire marker 50.

When the result of the detection of the marker 50 satisfies a conditionthat is determined in advance, the execution unit 104 executes a processthat is determined in advance. The process that is determined inadvance, which is executed by the execution unit 104 includes a processof transmitting predetermined information to a terminal. Thepredetermined information includes the alert information.

Moreover, the predetermined information, for example, may includeinformation indicating a likelihood that a large-scale theft will occur,a likelihood that stock-out will occur due to bulk buying or the like,or the like. Moreover, information that can identify a place (a shop, ashelf, or the like) and a point in time where these has occurred may beincluded.

Moreover, the predetermined information includes information thatidentifies an image obtained by image-capturing with the image capturecamera 10 that corresponds to the detected marker 50. The informationthat identifies the image, for example, includes at least one ofidentification information on the image capture camera 10, animage-capture point in time, and a file name of the image data 12.Furthermore, the image data 12 is configured with a plurality of frames.The information that identifies the image, for example, may furtherinclude a frame number at the time when the marker 50 is detected. Withthis configuration, because the frame at the time when the marker 50 isdetected can be identified, a manager can check the image data 12 thatprecedes the identified frame and the image data 12 that follows theidentified frame, and for example, can efficiently check whether or nota suspicious person appears in the image.

In an example of a processing condition 130 that is illustrated in FIG.7, a condition for detecting the marker 50, and each of a processing ina case where the condition is satisfied (True) and a processing in acase where the condition is not satisfied (False) are illustrated in amanner that is associated with each other.

In this example, a condition for detecting the marker 50 (hereinafteralso referred to as the marker (M002)) of a marker ID M002 is set to bea condition that an area (or the feature amount of the feature point)which accounts for a predetermined rate (for example, 80%) or greater ofthe entire area (or the feature amount of the feature point) of themarker (M002) is recognized by the image recognition processing. Thatis, in a case where an area (or the feature amount of the feature point)that accounts for more than 20% of the entire area (or the featureamount of the feature point) of the marker (M002) is not recognized bythe image recognition processing, it is determined that the detectioncondition is not satisfied.

Then, in a case where the condition is satisfied (True), notification isperformed, and in a case where the condition is not satisfied (False),it is determined that nothing is done.

At this point, for example, a message indicating that the marker 50 isdetected is notified. Alternatively, in another example, a name of thecommodity 42 that corresponds to the marker 50 or positional informationon the commodity shelving unit 40 may be notified in a manner that isincluded in the message. For example, the message may be displayed onthe monitor 24 of the server computer 20, or the message may betransmitted to the cash register 30 and the message may be displayed onthe shop-assistant display 32.

Destination information on a transmission destination may be stored in amanner that is associated with the processing condition 130. Aconfiguration may be employed in which a destination changes for eachdetection condition, and a configuration may be employed in whichtransmission to a predetermined destination is performed withoutdepending on the detection condition.

The processing condition 130 may be stored in a predetermined storageunit (not illustrated) and with reference to the storage unit at thetime of the image recognition processing the determination processingmay be performed. The determination processing according to theprocessing condition 130 may be included into a program.

The processing condition 130 may be determined for every marker 50, maybe determined for every type of the marker 50, may be determined forevery installation place of the marker 50, and may be determinedaccording to a combination of the type of marker 50 and the installationplace. The same processing condition 130 may be determined for differenttypes or different installation places.

The processing condition 130 will be described in more detail in theexample embodiments that will be described below.

A computer program that realizes the information processing apparatus100 which is configured in this manner will be described below.

The CPU 82 of the computer 80 in FIG. 4 reads the program 90, which isstored in the storage 85, into the memory 84 for execution, and thus canrealize each of the functions of the information processing apparatus100 according to the present example embodiment.

A computer program 90 according to the example embodiment of the presentinvention is written in such a manner that the computer 80 for realizingthe information processing apparatus 100 is caused to perform aprocedure for detecting the maker 50 that is determined in advance,which is different from the commodity 42, by the image recognitionprocessing on the image obtained by image-capturing the place (thecommodity shelving unit 40) where the commodity 42 is installed, and aprocedure for performing a predetermined process, when the result of thedetection of the marker 50 satisfies the condition that is determined inadvance.

The computer program 90 according to the present example embodiment maybe stored in a storage medium that is readable with the computer 80. Thestorage medium is not particularly limited and storage medium in variousforms are considered. Furthermore, the program 90 may be loaded from thestorage medium into the memory 84 of the computer 80, may be downloadedon the computer 80 through a network, and may be loaded into the memory84.

The media, on which the computer program 90 is stored, include anon-transitory tangible medium that is usable by the computer 80, andthe program code that is readable by the computer 80 is embedded in themedium. When the computer program 90 is executed on the computer 80, thecomputer 80 is caused to perform the following information processingmethod that realizes the information processing apparatus 100.

The information processing method of the information processingapparatus 100 according to the present example embodiment, which isconfigured in this manner, will be described below.

FIG. 8 is a flowchart illustrating an example of operation of theinformation processing apparatus 100 according to the present exampleembodiment.

An information processing method according to the example embodiment ofthe present invention is an information processing method of theinformation processing apparatus 100, and is an information processingmethod that is performed by the computer 80 which realizes theinformation processing apparatus 100.

The information processing method according to the present exampleembodiment includes detecting of the predetermine marker 50 by theinformation processing apparatus 100 performing the image recognitionprocessing on the image obtained by image-capturing the place where thecommodity 42 is placed (Step S103), and performing the predeterminedprocess, when the result of the detection of the marker 50 satisfies thecondition that is determined in advance (Step S107).

For more detail, the detection unit 102 performs the image recognitionprocessing on a certain frame or each frame of the image data 12, andextracts a feature (Step S101). Then, the detection unit 102 detects animage area that has a degree of similarity equal to or more than athreshold to a feature of the marker 50 which is registered in advance,as an image area of the marker 50 (YES in Step S103).

At this time, the detection unit 102 stores the marker ID of the marker50 and the detection date and time in the detection result storage unit120, in association with each other. Moreover, information on adetection place (for example, the commodity shelving unit 40, the shelf,a position within the shelf, a position within the captured image, orthe like) where the marker 50 is detected, and the identificationinformation on the image capture camera 10 may be included.

For example, as illustrated in FIG. 3A, the marker 50 is installed in apredetermined position on the commodity shelving unit 40. As illustratedin FIG. 3B, in a case where a plurality of commodities 42 are placed onthe commodity shelving unit 40, because the marker 50 is hidden by thecommodities 42, the marker 50 is not detected from the image data 12 (NOin Step S103). In this case, Step S105 and Step S107 are bypassed, andthe present processing is ended without anything being done.

Furthermore, as illustrated in FIG. 3B, a position in which thecommodity 42 has to be positioned and a relative position of the marker50 may be determined in advance and may be stored in the markerpositional-information storage unit 114. When this is done, from an areaof the marker 50 in view, it can be determined from which position thecommodity 42 disappears, in which position the commodity remains, whatpercentage of all the commodities 42 disappear or remain, or the like.

On the other hand, as illustrated in FIG. 3C, in a case where at leastsome of the commodities 42 disappear from the commodity shelving unit40, because at least a part of the marker 50 that is installed on thecommodity shelving unit 40 appears (In FIG. 3C, almost all thecommodities 42 disappear and the entire marker 50 appears), the marker50 is detected from the image data 12 (YES in Step S103). For example,in FIG. 8C, almost all commodities 42 disappear from the commodityshelving unit 40. Furthermore, as illustrated in FIG. 9, in a case whereonly some of the commodities 42 disappear from the commodity shelvingunit 40, because only a part of the marker 50 appears, a part of themarker 50 is detected from the image data 12 (YES in Step S103).

Then, the execution unit 104 determines whether or not the result of thedetection of the marker 50 satisfies a condition that is determined inadvance (Step S105). For example, in a case where a condition that isdescribed in the processing condition 130 in FIG. 7, that is, acondition that a detected a part accounts for a predetermined rate (forexample 80%) or greater of the maker (M002) is set and this condition issatisfied (True), the execution unit 104 notifies a destination that isdetermined in advance (here, the cash register 30) of a predeterminedmessage (Step S107).

In the present example embodiment, the predetermined message includesthe information that the marker (M002) is detected. Alternatively, thepredetermined message may include a name of the commodity 42 thatcorresponds to the marker (M002) or positional information (a floor, asales area, number of the commodity shelving unit 40, or the like) onthe place where the commodity 42 is placed. Moreover, the predeterminedmessage may include the information that unknown loss of the commodity42 occurs. At least one of these or a combination of these may beincluded. The predetermined message may include information such as anoccurrence point in time of the unknown loss.

For example, in the cash register 30, a message is received from theinformation processing apparatus 100 (the server computer 20), and thereceived message is displayed on the shop-assistant display 32.Accordingly, the shop assistant Ua can know that the maker (M002) isdetected (that the known loss occurs, or a place where the known lossoccurs).

Furthermore, in a case where the marker (M002) is detected (True), theexecution unit 104 may store the marker ID (M002) of the maker 50 thatis detected and the detection date and time information in the detectionresult storage unit 120. Moreover, in a case where the marker (M002) isnot detected (False), the information that the marker 50 is not detectedmay be stored in the detection result storage unit 120 in associationwith the date and time information. The storing of the information ofthe non-detection of the marker 50 may be performed for a predeterminedduration (for example, 30 minutes or the like).

Furthermore, as illustrated in FIG. 9, in the case where only some ofthe commodities 42 disappear from the commodity shelving unit 40,because only a part of the marker 50 appears, the part of the marker 50is recognized by the image recognition processing from the image data12. At this point, for example, approximately 60% of the marker 50 isassumed to be recognized. Therefore (due to the recognition ratio ofbelow 80%), the execution unit 104 determines that the marker (M002) isnot detected (NO in Step S105) and bypasses Step S107 and ends thepresent processing without anything being done.

At this time, the execution unit 104 may further store the marker ID(M002) of the recognized marker 50, the date and time information, andthe information that approximately 60% of the marker 50 is recognized,in the detection result storage unit 120 in association with each other.

As described above, in the information processing apparatus 100according to the present example embodiment, the marker 50 that isdetermined in advance is detected by the detection unit 102, byperforming the image recognition processing on the image obtained byimage-capturing the place where the commodity 42 is placed, and when theresult of the detection of the marker 50 satisfies the condition that isdetermined in advance, a process that is determined in advance isexecuted by the execution unit 104.

In this manner, in the information processing apparatus 100 according tothe present example embodiment, the marker 50 that is determined inadvance is installed on the place (the commodity shelving unit 40) onwhich the commodity 42 is placed, the image recognition processing isperformed on the image obtained by image-capturing the place, and themarker 50 is detected. Only by doing this, an effect is achieved inwhich the unknown loss, such as the disappearance of a large number ofcommodities 42, can be simply detected and can be notified.

A difference between time-series changes in the image data 12 obtainedby image-capturing the commodity shelving unit 40 can be detected andthe disappearance of the commodity 42 from the commodity shelving unit40 can be detected, but in such a case, there is a problem in that loadon image processing is raised. In the information processing apparatus100 according to the present example embodiment, because only with theresult of the detection of the marker 50, the unknown loss can bedetected to be notified, the load on the image processing is lowered.Thus, the efficiency is increased and the cost relating to the imageprocessing is also suppressed to a low level.

Second Example Embodiment

Next, an information processing apparatus 200 according to a secondexample embodiment of the present invention will be described.

FIG. 10 is a functional block diagram illustrating a logicalconfiguration of the information processing apparatus 200 according tothe second example embodiment of the present invention.

The information processing apparatus 200 according to the presentexample embodiment includes a detection unit 202, an execution unit 204,and an identification unit 206. The detection unit 202 and the executionunit 204 have functions similar to those of the detection unit 102 andthe execution unit 104, respectively, of the information processingapparatus 100 in FIG. 1 and, along with this, have functions that willbe described below.

Note that a configuration of the information processing apparatus 200according to the present example embodiment can be combined with aconstitution of an information processing apparatus according to anotherexample embodiment, which will be described below.

The information processing apparatus 200 is connected to a storageapparatus 210 in an accessible manner. The storage apparatus 210includes a storage unit similar to the storage apparatus 110 accordingto the example embodiment, which is described above. The storageapparatus 210, for example, is a hard disk, a Solid State Drive (SSD), amemory card, or the like, and may be included in the informationprocessing apparatus 200 and may be an external apparatus.

In the information processing apparatus 200 according to the presentexample embodiment, a plurality of markers 50 are set to be on the placewhere the commodity 42 is placed.

The identification unit 206 identifies a type of, or a place of, themarker 50 that is detected by the detection unit 202.

The execution unit 204 executes a process in accordance with theidentified type of, or the identified place of, the marker 50.

Furthermore, the execution unit 204 may perform processing in accordancewith the combination of the detected markers 50.

Types of markers 50 are considered in various ways, such as is the casewhen a distinct by a common feature or quality is made, when a commonproperty is shared, and when a difference is recognized. A type that isidentified may be registered in advance. It may be identified which oneis a corresponding type. The identification unit 206 may divide typesaccording to features of a plurality of detected markers 50. Forexample, in a case where there are a red marker 50 in the shape of acircle, a black marker 50 in the shape of a quadrangle, and a bluemarker 50 of a logotype, the markers 50 may be divided by a figure (afigure such as a circle or quadrangle, and a logotype) into two types,and may be divided by a tone of color, for example, into two types,color, and black and white.

A place of the marker 50 is identified by the positional information onthe marker 50 described above. For example, the place of the marker 50can be identified by the store floor, the area, the commodity shelvingunit 40, the position of the shelf, the position on the shelf, theposition of the image capture camera 10, the image capture range of theimage capture camera 10, or the like. Moreover, in a case where aplurality of makers 50 are included in the image capture range of theimage capture camera 10, the place of the maker 50 may be identified byinformation including positional information relative to any othermarker 50 or to a position that is a predetermined reference.

An example in which processing varies according to a type of the marker50 will be described below with reference to FIGS. 11 and 12.

FIG. 11A illustrates a scene in which two types, a marker 50 a and amarker 50 b are pasted on the shelf plate 44. The marker 50 a is made byprinting a logotype of a brand ABC. The marker 50 b is made by printinga logotype of a brand XYZ.

A marker ID of the marker 50 a is assumed to be M004, and a marker ID ofthe marker 50 b is assumed to be M006.

At places where respective markers 50 are pasted on the shelf plate 44,clothes with a brand that corresponds to each logotype are assumed to bestacked on top of one another. In FIG. 11B, commodities 42 a that areclothes with the brand ABC are assumed to be stacked on top of oneanother on the marker 50 a, and commodities 42 b that are clothes withthe brand XYZ are assumed to be stacked on top of one another on themarker 50 b.

In FIG. 11B, because each commodity 42 is placed on each marker 50, eachcommodity 42 is captured in the image data 12, and the marker 50 is notcaptured (the marker 50 is indicated by a broken line). Consequently,the detection unit 102 does not detect any marker 50.

In FIG. 11C, a scene in which only the commodity 42 a with the brand ABCremains on the shelf plate 44 and the commodity 42 b with the brand XYZdisappears is illustrated. The commodity 42 a and the marker 50 b arecaptured in the image data 12, and the marker 50 a is not captured (themarker 50 a is indicated by a broken line). Consequently, the detectionunit 102 detects only the marker 50 b, and does not detect the marker 50a.

FIG. 12 is a diagram illustrating an example of a processing condition232 of the information processing apparatus 200 according to the presentexample embodiment.

As described in the processing condition 232, in this example, a brandname that corresponds to the marker 50 is notified according to the type(here, the marker ID) of the marker 50 that is detected from the imagedata 12, in a manner that is included in the message. Accordingly, whichbrand the commodity 42 of which the unknown loss occurs has can besimply notified.

In an example of a modification to this example, for example, if abarcode in which commodity information is stored is used as the marker50, a configuration can be employed in which when the marker 50 isdetected, the barcode is read and decrypted and the commodityinformation is notified.

Furthermore, in the present example embodiment, the identification unit206 may identify a range which corresponds to the detected part of themarker 50. The execution unit 204 may perform a process in accordancewith the range that is identified by the identification unit 206.

In the present example embodiment, as illustrated in FIG. 13A,consecutive numeral symbols 1 to 7 are included in predeterminedpositions of the marker 50, and thus, based on the numeral symbols thatare recognized by the detection unit 102, a range corresponding to thenumeral symbols can be identified as the detected range.

In an example in FIG. 13A, the numeral symbol 1 corresponds to theforemost position on the shelf plate 44. The greater numeral, the morebackward the position corresponding to the numeral symbol is locatedtoward the rear of the shelf plate 44. The numeral symbol 7 correspondsto the rearmost position on the shelf plate 44. Here, the range that isrecognized within the marker 50 and identified corresponds to a range inwhich the commodity 42 disappears.

In an example in FIG. 13B, the marker 50 that includes the consecutivenumeral symbols 1 to 6 in predetermined positions are pasted on a rearsurface plate 46 of the commodity shelving unit 40. The numeral symbol 1corresponds to a position that is closest to the shelf plate 44. Thegreater numeral, the more upward the position corresponding to thenumeral symbol is located in the height direction from the shelf plate44. The numeral symbol 6 corresponds to the uppermost position from theshelf plate 44. Here, the range that is recognized within the marker 50and identified, corresponds to a range in which the commodities 42 thatare stacked on top of one another on the shelf plate 44 disappear.

In an example in FIG. 13A, the image capture camera 10 is installed in aposition in the upward direction, which faces an upper surface of theshelf plate 44 of the commodity shelving unit 40 on which the marker 50is installed, and performs image capture toward the direction of theshelf plate 44 of the commodity shelving unit 40.

In an example in FIG. 13B, the image capture camera 10 is installed in aposition in the horizontal direction, which faces a surface of the rearsurface plate 46 of the commodity shelving unit 40 on which the marker50 is installed, and performs the image capture toward the direction ofthe rear surface plate 46 of the commodity shelving unit 40.

The execution unit 204 may change processing according to a range of themarker 50 that is identified in this manner. In the present exampleembodiment, processing that corresponds to the numeral indicating adetected range of the marker 50 is defined in advance by a processingcondition 230 that will be described below.

FIG. 14A is a diagram of the shelf plate 44 in FIG. 13A, when viewedfrom above.

FIG. 14B illustrates a scene in which many commodities 42 are on displayon the commodity shelving unit 40. In this manner, in a case where manycommodities 42 are on display on the shelf plate 44 of the commodityshelving unit 40, almost the entire marker 50 is hidden by thecommodities 42. Consequently, the detection unit 102 does not recognizethe marker 50 by the image recognition processing.

FIG. 14C illustrates a scene in which several of commodities 42disappear and thus a part of the marker 50 appears. At this point, amongthe numeral symbols 1 to 7 on the marker 50, only 1 comes into view.Consequently, the detection unit 102 recognizes the numeral 1 on themarker 50 by the image recognition processing. The identification unit206 identifies the range of the marker 50 as a 0-to-1 range of the nearside 44 a of the shelf plate 44.

FIG. 14D illustrates a scene in which approximately half of thecommodities 42 disappear and thus half of the marker 50 appears. At thispoint, among the numeral symbols 1 to 7 on the marker 50, the numeral 1to 4 come into view. Consequently, the detection unit 102 recognizes thenumerals 1 to 4 on the marker 50. The identification unit 206 identifiesa 2-to-5 range of the marker 50 as a range from the near side 44 a tothe middle of the shelf plate 44.

In this manner, with the range where a plurality of markers 50 (at thispoint, each numeral within the marker 50 also serves as positionalinformation) that are arranged toward the depth direction of the shelfplate 44 of the commodity shelving unit 40 come into view, approximatenumerical quantity of commodities 42 that disappear can be estimated.For example, in a case where the numerals of up to 7 on the marker 50are recognized, a state can be estimated where many commodities 42disappear.

FIG. 15 is a diagram illustrating an example of the condition 230 forprocessing by the information processing apparatus 200 according to thepresent example embodiment.

As described in the processing condition 230, in the present exampleembodiment, from the result of the detection of the marker (M003) inFIG. 14A, for example, depending the visibility (which corresponds tothe detected ranges of the maker 50) of the numeral symbols 1 to 7 thatare written on the marker (M003), an alarm level at which notificationis performed is caused to change. That is, processing is changedaccording to the detected range of the detected marker 50. In theprocessing condition 230, each range to which a maximum value (forexample, in a case where the numerals 1 to 5 come into view, a maximumvalue is 5) among the numerals that are detected is assigned is definedas the detected range.

For example, in a case where no numeral symbols on the maker come intoview (0) or in a case where the numeral symbol 1 comes into view (isrecognized by the detection unit 102), an alarm is set to a level 1. Itis assumed that the greater numeral, the higher a warning level. Then,in a case where any one of the numerals 2 to 5 on the marker comes intoview (is recognized by the detection unit 102), the alarm level is setto 2. Moreover, in a case where at least one of the numerals 6 and 7 onthe marker comes into view (is recognized by the detection unit 102),the alarm level is set to 3. In this example, the execution unit 104outputs the alarm level that is set in a case where the result of thedetection satisfied a condition, as a message.

That is, in a case shown in FIG. 14B where no numerals on the marker aredetected from the image data 12, an alarm level 1 that corresponds tothe 0-to-1 detected range is set and the message is output.

That is, in a case shown in FIG. 14C where the numeral 1 on the markeris detected from the image data 12, the alarm level 1 is set, and themessage is output.

In a case shown in FIG. 14D where the numerals 1 to 4 on the marker aredetected from the image data 12, an alarm level 2 that corresponds tothe 2-to-5 detected range, in which the maximum value 4 is included isset, and the message is output.

In a case shown in FIG. 14A where the numerals 1 to 7 on the marker aredetected from the image data 12, an alarm level 3 that corresponds to a6-to-7 detected range, in which the maximum value 7 is included is set,and the message is output.

Furthermore, a destination to which the message is transmitted may bechanged according to the alarm level. The destination information isseparately stored in advance in a destination table (not illustrated) inan associated manner. Then, the execution unit 204 may acquire thedestination information that corresponds to the alarm level, from thedestination table, and may transmit the acquired destinationinformation.

The detection unit 202 according to the present example embodiment has afunction similar to that of the information processing apparatus 100according to the example embodiment in FIG. 1, which is described above,and, along with this, detects the marker 50 using the feature of themarker 50 that is learnt by machine learning.

The machine learning, for example, can be realized by a machinelearning-type detection engine.

The feature of the marker 50 from the image data on the marker 50 islearnt by the machine learning-type detection engine, and the feature isregistered in a leaning library (equivalent to the marker informationstorage unit 112). Moreover, the marker 50 that is learnt is registeredin the marker positional-information storage unit 114 in a manner thatis associated with information on the installation place of the marker50. Furthermore, the information on the installation place (an image orinformation that can identify a place is used) may also be learnt by themachine learning, and may be registered in the learning library(equivalent to the marker positional-information storage unit 114).

For example, states in FIGS. 14A to 14D may be individually learnt bythe machine learning, shapes or visibilities (partial, whole, or theposition or the like) of the marker (M002) may be divided according tocases, a result of the division may be presented to the manager, and themanager may register the processing condition 230 in an associatedmanner. Moreover, the marker 50 (for example, the barcode) or the likethat indicates positional information (information that identifies afloor, a shop, and a shelf of the commodity shelving unit 40) on theshelf plate 44 is further installed on the shelf plate 44, and isimage-captured by the image capture camera 10 together with the marker(M002). Thus, the positional information is acquired from the barcode,and may be stored, as learnt positional information on the maker (M002),in the marker information storage unit 112 in an automaticallyassociated manner.

A computer program for the information processing apparatus 200according to the present example embodiment, which is configured in thismanner, will be described below.

The computer program according to the present example embodiment iswritten in such a manner that the computer 80 for realizing theinformation processing apparatus 200 is caused to execute a procedurefor identifying the type of, or the place of, the detected marker 50 anda procedure for performing processing in accordance with the identifiedtype of, or the identified place of, the marker 50.

The information processing method of the information processingapparatus 200 according to the present example embodiment, which isconfigured in this manner, will be described below.

FIG. 16 is a flowchart illustrating an example of operation of theinformation processing apparatus 200 according to the present exampleembodiment.

The information processing apparatus 200 according to the presentexample embodiment includes a procedure (Step S101, Step S103, and StepS107) similar to those in the flowchart in the information processingapparatus 100 in FIG. 8 according to the example embodiment, which isdescribed above, and further includes Step S201 and Step S203.

An information processing method according to the example embodiment ofthe present invention is an information processing method of theinformation processing apparatus 200, and is an information processingmethod that is performed by the computer 80 which realizes theinformation processing apparatus 200.

The information processing method according to the present exampleembodiment includes the information processing apparatus 200'sidentifying the type of, or the place of (or the range), the detectedmarker 50 (Step S201), and the information processing apparatus 200'sexecuting the process in accordance with the identified type, or theidentified place of, the marker 50 (Step S107).

An example in which processing varies according to a range of the markerwill be described in detail below with reference to FIGS. 14 to 16.

As described with reference to FIG. 8, in Step S103, the detection unit202 detects an image area that has a degree of similarity equal to ormore than a threshold to a feature of the marker 50 which is registeredin advance, as an image area of the marker 50 (YES in Step S103).

Moreover, the identification unit 206 identifies the range of the marker50 that is detected (Step S201). Here, numerals within the marker 50 arerecognized by the image recognition processing. For example, in the caseof FIG. 14D, the numerals 1 to 4 are identified.

Then, the execution unit 204 refers to the processing condition 230, andbecause the identified range satisfies the condition 2-to-5 (YES in StepS203), the execution unit 204 executes setting to the alarm level 2, asthe corresponding process, and executes a process of notifying a messageof a predetermined terminal (Step S107).

Furthermore, in the case of FIG. 14C, in Step S201, the identificationunit 206 identifies the numeral 1 as the range of the marker 50. Then,because the processing condition 230 is referred to and the identifiedrange satisfies the condition 0-to-1 (YES in Step S203), the executionunit 204 executes setting to the alarm level 1, as the correspondingprocess, and executes a process of notifying a message of apredetermined terminal (Step S107).

Moreover, in the case of FIG. 14A, in Step S201, the identification unit206 identifies the numerals 1 to 7 as the range of the marker 50. Then,the execution unit 204 refers to the processing condition 230, andbecause the identified range satisfies the condition 6-to-7 (YES in StepS203), the execution unit executes setting to the alarm level 3, as thecorresponding process, and executes a process of notifying a message ofa predetermined terminal (Step S107).

In this manner, according to the present example embodiment, a processthat is determined in advance can be performed according to at least oneof the range, the type, and the place of the marker 50. As illustratedin FIG. 14C, in a case where a small number of commodities 42 disappear,the alarm level 1 is notified. As illustrated in FIG. 14D, in a casewhere half of the commodities 42 disappear, the alarm level 2 isnotified. As illustrated in FIG. 14A, in a case where all of thecommodities 42 disappear, the alarm level 3 is notified.

As described above, in the information processing apparatus 200according to the present example embodiment, at least one of the type,the place, and the range of the marker 50 that is detected by theidentification unit 206 is identified, and processing is performedaccording to at least one of the type, the place, and the range, whichare identified, and a combination of these. With this configuration, thesame effect as in the example embodiments described above, and, alongwith this, various types of processing are possibly performed in anassociated manner according to the type, the place, and the range of themarker 50.

Third Example Embodiment

Next, an information processing apparatus 300 according to a thirdexample embodiment of the present invention will be described.

FIG. 17 is a functional block diagram illustrating a logicalconfiguration of the information processing apparatus 300 according tothe third example embodiment of the present invention.

The information processing apparatus 300 according to the presentexample embodiment is different from the example embodiments describedabove in that the result of the detection of the marker 50 is stored ina time-series manner and in that processing is changed according to adifference between the previously detected marker 50 and the currentlydetected marker 50.

The information processing apparatus 300 is further connected to thestorage apparatus 310 in an accessible manner. The storage apparatus 310includes a storage unit similar to at least one of the storage apparatus110 and the storage apparatus 210 according to the example embodimentsdescribed above. The storage apparatus 310, for example, is a hard disk,a Solid State Drive (SSD), a memory card, or the like, and may beincluded in the information processing apparatus 300 and may be anexternal apparatus.

A configuration of the information processing apparatus 300 according tothe present example embodiment can be suitably combined with theconfigurations of the information processing apparatus according to theother example embodiments of the present invention, within a range wherethere is no contradiction.

The information processing apparatus 300 according to the presentexample embodiment includes a detection unit 302 and an execution unit304. The information processing apparatus 300 may further includeconfiguration similar to that of the information processing apparatus200 in FIG. 10 according to the example embodiment, which is describedabove.

The detection unit 302 has a function similar to that of the detectionunit 102 according to the example embodiment described above, stores theresult of the detection of the detected marker 50 in a time-seriesmanner in the storage apparatus 310, and obtains the difference betweenthe previously detected marker 50 and the currently detected marker 50.

The execution unit 304 executes a process in accordance with thedifference between the markers 50 that are detected by the detectionunit 302.

In the present example embodiment, the detection unit 302 is assumed toperform detection processing of the marker 50 with a predeterminedperiod, and for example, all results of the detection may be stored.Alternatively, a configuration may be employed in which only in a casewhere a change in the feature amount equal to or more than a thresholdfrom the result of the previous detection is detected, the result of thedetection is stored.

In the present example embodiment, the latter case is employed. That is,only in the case where the change in the feature amount equal to or morethan the threshold from the result of the previous detection isdetected, the result of the detection is stored.

In the present example embodiment, the terms “previous” and “current”refer to the results of the detection at two points in time, betweenwhich there is a predetermined duration. The term “previous” is assumedto refer to earlier (older) information than the term “current.” Theterm “current” may not refer to the present time, and may refer to acertain point in the past (in history information).

The difference that is detected by the detection unit 302 is consideredin various ways, and examples thereof are given as follows. However, nolimitation to these is imposed. Furthermore, the followings can becombined.

(b1) Difference between areas of the marker area that is detected

(b2) Difference between the numbers of predetermined objects (forexample, letters, marks, or the like) that are included in the markerthat is detected

(b3) Difference between maximum values or minimum values (or ranges of)among predetermined numerals that are included in the marker that isdetected

(b4) Difference between the numbers of types of the marker that isdetected

A timing at which the difference is obtained is considered in variousways, examples thereof are given as follows. However, no limitation tothese is imposed. Furthermore, the followings can be combined.

(c1) Every period of the detection processing

(c2) Whenever the result of the detection is stored

(c3) Every predetermined time (every five minutes or the like), everypoint in time (every hour or the like), or the like

At this point, the examples (b3) and (c2), which are described above,will be described below.

As illustrated in FIG. 13B, in the present example embodiment, themarker 50 in which the numerals 1 to 6 are lined up from downward toupward is installed on the rear surface plate 46 of the commodityshelving unit 40.

FIG. 19 is a diagram illustrating an example of data structure of adetection result storage unit 320 according to the present exampleembodiment.

In this example, the result of the detection for every shelf plate 44 ofthe commodity shelving unit 40 is stored, but no limitation to this isimposed. The result of the detection of the marker 50 on the shelf plate44 (T1001) is stored in a detection result storage unit 320 a.

In FIG. 19, a maximum value (or a minimum value) among the numerals onthe marker detected by the detection unit 302, along with the detectiondate and time information, is stored in the detection result storageunit 320 a or a detection result storage unit 320 b for every shelfplate 44.

As illustrated in FIG. 18A, in a case where the commodities 42 fullyoccupy the commodity shelving unit 40, the marker 50 is hidden by thecommodities 42 from view. The maker 50 is almost not captured into theimage data 12 obtained by image-capturing with the image capture camera10. Consequently, the detection unit 302 does not detect the marker 50from the image data 12. In a case where nothing is detected, the numeral0 is stored in the detection result storage unit 320 a (FIG. 19A) of thedetection result storage unit 320 in a manner that is associated withpoint-in-time information. Furthermore, in other example embodiments, aconfiguration may be employed in which nothing is stored in thedetection result storage unit 320.

Next, as illustrated in FIG. 18B, when several commodities 42 disappear,a part of an upper part of the marker 50 comes into view. A part of themarker 50, the numeral 6 is captured into the image data 12 obtained byimage-capturing with the image capture camera 10. Consequently, thedetection unit 302 detects a part of the marker 50 from the image data12 and recognizes the numeral 6.

Then, the detection unit 302 stores the numeral 6 in the detectionresult storage unit 320 a (FIG. 19A) in a manner that is associated withthe detection date and time information. In the present exampleembodiment, a minimum value of the recognized numeral is stored in theresult of the detection. Depending on the order of numerals that arewritten on the marker 50 and a way of placing the commodities 42 ondisplay, it can be determined which one of a minimum value and a maximumvalue among numerals, and a range of numerals is stored. In the presentexample embodiment, as the number of commodities 42 decreases, anappearing numeral becomes gradually smaller from 6 to 1. Because ofthis, a minimum value of the numerals is used as an index indicating astate where the number of commodities 42 decreases.

On the other hand, in the case of the marker 50 that is illustrated inFIG. 14A, as the number of commodities 42 decreases, an appearingnumeral becomes gradually larger from 1 to 7. Because of this, a maximumvalue of the numerals is used as an index indicating a state where thenumber of commodities 42 decreases.

First, normal commodity sales time will be described. In this case,because the number of commodities 42 gradually decreases, the numerals 5and 4 are sequentially detected with the passage of time, and, asillustrated in FIG. 19A, are stored in the detection result storage unit320 a in a manner that is associated with detection-point-in-timeinformation. Moreover, for example, commodities 42 are additionallysupplied and so forth at 13:00, and thus the marker is completelyhidden. Because of this, the marker is not detected. Accordingly, thenumeral 0 is stored in the detection result storage unit 320 a (FIG.19A) of the detection result storage unit 320 in a manner that isassociated with the point-in-time information.

Next, a case where the products disappear abruptly, such as is the casewhen the large-scale theft or the bulk buying occurs, will be described.

First, because of a state (FIG. 18A) where the commodities 42 fullyoccupy the commodity shelving unit 40, the marker is not detected.Accordingly, the numeral 0 indicating that nothing is detected is storedin the detection result storage unit 320 b (FIG. 19B) of the detectionresult storage unit 320 in a manner that is associated with thepoint-in-time information.

Then, when some time elapsed, the number of commodities 42 decreases,and, as illustrated in FIG. 18A, the numeral 6 appears. Because of this,the numeral 6 is recognized and the numeral 6 is stored in the detectionresult storage unit 320 b (FIG. 19B).

Then, moreover, when some time elapsed (for example, two minutes afterthe previous marker is detected), a large number of products are assumedto disappear at one time as illustrated in FIG. 18D. At this time, withthe detection unit 302, the numerals 1 to 6 are recognized, and thenumeral 1 that is a minimum value is stored in the detection resultstorage unit 320 b (FIG. 19B).

In the present example embodiment, the detection unit 302, as describedabove, stores a result of the detection of the detected marker 50 in atime-series manner in the detection result storage unit 320. Then, thedetection unit 302 further refers to the detection result storage unit320 and thus obtains the difference between the previously detectedmarker 50 and the currently detected marker 50.

Obtained differences, as illustrated in FIG. 19, can be stored in eachof the detection result storage unit 320 a and the detection resultstorage unit 320 b.

Then, the execution unit 304 executes a process in accordance with thedifference between the detected markers 50.

FIG. 20 is a diagram illustrating an example of a condition 330 forprocessing by the information processing apparatus 300 according to thepresent example embodiment.

As illustrated in the processing condition 330, in the present exampleembodiment, association of processing in accordance with the differencebetween the markers is made. For example, in the example, a process in acase where a difference in the minimum value of the numerals on thedetected marker satisfies a condition, that is, a detection conditionthat a difference is equal to or smaller than −5 is determined.

In a case where the difference is smaller than −5, notification isperformed. In a case where the condition is not satisfied, the processis not performed on anything.

In the present example embodiment, in a case where a change is detectedby the image recognition processing that is performed with apredetermined period, a result of the detection is stored and adifference for every stored result of the detection is obtained.However, in a case where the result of the detection is stored with thepredetermined period, a timing at which the difference is calculated andis stored, for example, can be set with a predetermined period (forexample, every five minutes, or the like). Then, in a case where thestoring is performed at this timing, for example, not only may adifference for every five minutes be obtained, but a difference (a sumof the results of the detection that is performed a predetermined numberof times) for every 30 minutes may also be obtained. Thus, thesedifferences may be defined as the detection conditions.

Furthermore, in another example of the processing condition 330, in acase where a change in a difference equal to or more than apredetermined value occurs within a short time, an alarm level may beset to be high and thus notification may be made. That is, alarm levelsmay be set in a step-by-step manner using the time of a change with thedifference equal to or more than the predetermined value as a condition.

A computer program for the information processing apparatus 300according to the present example embodiment, which is configured in thismanner, will be described below.

The computer program according to the present example embodiment iswritten in such a manner that the computer 80 for realizing theinformation processing apparatus 300 is caused to execute a procedurefor storing the result of the detection of the detected marker 50 in atime-series manner in the storage apparatus 310, a procedure forobtaining a difference between the previously detected marker 50 and thecurrently detected markers 50, and a procedure for performing processingin accordance with the difference between the detected markers 50.

The information processing method of the information processingapparatus 300 according to the present example embodiment, which isconfigured in this manner, will be described below.

FIG. 21 is a flowchart illustrating an example of operation of theinformation processing apparatus 300 according to the present exampleembodiment.

The information processing apparatus 300 according to the presentexample embodiment includes a procedure (Step S101 to Step S107) similarto those in the flowchart of the information processing apparatus 100 inFIG. 8 according to the example embodiment, which is described above,and further includes Step S301 and Step S303.

An information processing method according to the example embodiment ofthe present invention is an information processing method of theinformation processing apparatus 300, and is an information processingmethod that is performed by the computer 80 which realizes theinformation processing apparatus 200.

The information processing method according to the present exampleembodiment includes the information processing apparatus 300 storing theresult of the detection of the detected marker 50 in a time-seriesmanner in the storage apparatus 310 (Step S301), the informationprocessing apparatus 300 obtaining the difference between the previouslydetected marker 50 and the currently detected marker 50 (Step S303), andthe information processing apparatus 300 performing the process inaccordance with the difference between the detected markers 50 (StepS105 and Step S107).

The detail will be described below.

First, an example of an operation at the time of normal commodity saleswill be described.

First, as described with reference to FIG. 8, in Step S103, thedetection unit 302 detects an image area that has the degree ofsimilarity equal to or more than a threshold to a feature of the marker50 which is registered in advance, as an image area of the marker 50(YES in Step S103).

For example, as illustrated in FIGS. 18A and 18B, in a case where thenumber of commodities 42 decreases gradually with the passage of time,the result of the detection, as illustrated in the detection resultstorage unit 320 a in FIG. 19A, is obtained.

The detection unit 302 obtains the difference between the previouslydetected marker 50 and the currently detected marker 50 (Step S303) andstores the obtained difference in the detection result storage unit 320a.

Then, the execution unit 304 refers to the processing condition 330, andbecause the difference between markers 50 is neither equal to norsmaller than −5 (NO in Step S105), the execution unit 304 bypasses StepS107 and ends the present processing without performing any processing.

Next, an example of an operation in a case where the products disappearabruptly, such as is the case when the large-scale theft or the bulkbuying occurs, will be described.

First, as described with reference to FIG. 8, in Step S103, thedetection unit 302 detects an image area that has the degree ofsimilarity equal to or more than a threshold to a feature of the marker50 which is registered in advance, as an image area of the marker 50(YES in Step S103).

For example, in a case where a change occurs from a state in FIG. 18B toa state in FIG. 18D for approximately two minutes, the result of thedetection is obtained as illustrated in the detection result storageunit 320 b in FIG. 19B.

The detection unit 302 obtains the difference between the previouslydetected marker 50 and the currently detected marker 50 (Step S303) andstores the obtained difference in the detection result storage unit 320b.

Then, referring to the processing condition 330, the execution unit 304determines that the detection condition that the difference betweenmarkers 50 is equal to or smaller than −5 is satisfied (YES in StepS105), executes a process that notifies a predetermined terminal of amessage or the like indicating that a large number of commodities 42disappear (Step S107), and ends the present processing.

In this manner, according to the present example embodiment, thedifference between the previously detected marker 50 and the currentlydetected marker 50, and according to the difference, a process that isdetermined in advance can be performed. In a case where the number ofcommodities 42 decreases gradually such as when the commodity is soldnormally, message notification processing is not performed. In a casewhere a large number of commodities 42 disappear at one time, themessage notification processing is performed.

As described above, in the information processing apparatus 300according to the present example embodiment, the difference between thepreviously detected marker 50 and the currently detected marker 50 isobtained by the detection unit 302, and the process in accordance withthe difference is performed by the execution unit 304.

With this confirmation, the same effect as in the example embodimentsdescribed above is achieved, and along with this, the processing can bechanged according to temporal changes of marker 50. Because of this, itis detected that a large number of commodities 42 disappear within ashort time, and thus reporting is possible. Therefore, in theinformation processing apparatus 300 according to the present exampleembodiment, with a simple configuration, the large-scale theft, the bulkbuying, or the like can be detected and can be notified.

The example embodiments of the present invention are described abovewith reference to the drawings, but these are only examples of thepresent invention, and various configurations other than theconfigurations described above can be employed.

For example, a reference marker 60 that serves as a reference fordetermining a target area where a marker is to be detected by adetection unit may be installed independently of the marker 50.

The detection unit 102, 202, or 302 detects the reference maker 60, andperforms the detection processing of the marker 50 over a detectiontarget area which corresponds to the detected reference marker 60.

The detection area is assumed to be defined in advance for eachreference maker 60. First, the reference maker 60 is detected by thedetection unit, the detection processing of the marker 50 is performedover the detection target area that corresponds to the detectedreference marker 60. Furthermore, a processing condition may be providedfor each detection area. That is, the identical marker 50 may beincluded in different detection areas respective corresponding todifferent reference markers 60, and processing in accordance with aprocessing condition that differs from one detection area to another maybe performed.

As illustrated in FIG. 22, two types of markers, a marker 50 a and amarker 50 b are assumed to be installed on the shelf plate 44.

An area 62 a where a reference maker 60 a is detected is indicated by abroken line, and an area 62 b where a reference marker 60 b is indicatedby an alternate long- and short-dashed line.

Furthermore, in a case where the same reference markers 60 (for example,the reference marker 60 a) are installed on shelf plates 44 of differentcommodity shelving units 40, the detection of the markers 50 may beperformed on the different commodity shelving units 40 using thedetection area 62 a of the reference marker 60 a. Furthermore, ondifferent commodity shelving units 40, the same processing condition maybe used and different processing conditions may be used over thedetection area 62 a of the same reference marker 60 a. For example, evenfor detection areas 62 of the same reference marker 60, the processingcondition may also be changed depending on a place such as the commodityshelving unit 40 or the floor.

With this configuration, the detection processing of the marker 50 canbe performed with the detection area being narrowed. Along with this,even for the same marker 50, a different processing condition can alsobe set and processing different depending on the condition can beperformed.

EXAMPLE Example 1

FIG. 23 is a diagram conceptually illustrating an example of a systemconfiguration of an example of an information processing apparatus 400according to the present invention.

In this example, the information processing apparatus 400 has an imageacquisition function unit 402, a detection engine control function unit404, a marker learning function unit 406, and a result reading viewer408.

The image acquisition function unit 402 may acquire captured-image data(image data 424) at any time from an IP camera 420, and may read andacquire an image file 422 obtained by image-capturing in advance andstored in a storage apparatus (not illustrated), at any timing.

The image data 424 that is acquired by the image acquisition functionunit 402 is delivered to the detection engine control function unit 404.

The detection engine control function unit 404 further includes amachine learning-type detection engine 410 and a setting function unit412. The marker learning function unit 406 learns the marker 50 using astill image data 430 obtained by image-capturing the marker 50, andthus, a learning library 432 that is accordingly generated is deliveredto and is registered in machine learning-type detection engine 410.

Moreover, regarding information on the marker 50, which is registered inthe learning library 432, the setting function unit 412 receives settingof a corresponding process, and registers the setting in the learninglibrary 432. That is, the setting function unit 412 has a function ofsetting various processing conditions, which are described above in theexample embodiment.

The detection engine control function unit 404 is equivalent to each ofthe detection units according to the example embodiments, which aredescribed above. The marker 50 is detected from the image data 424, andresult data 434, which results from the detection, is delivered to theresult reading viewer 408. At this point, as the result of thedetection, a message (predetermined information), which is set by eachof the execution units according to the example embodiments, which aredescribed above, may be included in the result data 434. Through theresult reading viewer 408, detection result information is provided tothe manager in a readable manner. For example, the detection resultinformation is displayed on the monitor 24 of the server computer 20 orthe shop-assistant display 32 of the cash register 30 in FIG. 2.

The invention in the present application is described above withreference to the example embodiments and the example, but the inventionin the present application is not limited to the example embodiments andthe example. Various changes to the configuration or the detail of theinvention in the present application, which can be apparent to a personskilled in the art, can be made within the scope of the invention in thepresent application.

Note that, in a case where information relating to a user is acquiredand used according to the present invention, the acquisition and the useare supposed to be lawfully performed.

Examples of reference example embodiment will be added as appendixesbelow.

1. An information processing apparatus including a detection unit thatdetects a difference from a previously detected marker in apredetermined marker, by image recognition processing on an imageobtained by image-capturing an arrangement place of an article; and anexecution unit that executes a process in accordance with thedifference.

2. The information processing apparatus according to 1, in which aplurality of the markers are set on the place, and in which thedetection unit stores a result of the detection of the detected markerin a storage apparatus in a time-series manner and obtains a differencebetween the previously detected marker and the currently detectedmarker.

3. The information processing apparatus according to 1 or 2, furtherincluding: an identification unit that identifies a type or a place ofthe detected marker, in which a plurality of the markers are set on theplace, and in which the execution unit executes a process in accordancewith the identified type or the identified place of the marker.

4. The information processing apparatus according to 3, in which aplurality of the markers are set on the place, and in which theexecution unit executes a process in accordance with a combination ofthe detected markers.

5. The information processing apparatus according to 3 or 4, in whichthe identification unit further identifies a detected range of themarker, and in which the execution unit executes a process in accordancewith the range that is identified by the identification unit.

6. The information processing apparatus according to any one of 1 to 5,in which the detection unit detects a marker using a feature of themarker that is learnt by machine learning.

7. The information processing apparatus according to any one of 1 to 6,in which the predetermined process, which is executed by the executionunit includes a process of transmitting predetermined information to apredetermined terminal.

8. The information processing apparatus according to 7, in which thepredetermined information includes information that identifies an imageobtained by image-capturing by an image capture unit that corresponds tothe detected marker.

9. The information processing apparatus according to 7 or 8, in whichthe predetermined information includes alert information.

10. The information processing apparatus according to any one of 1 to 9,in which the predetermined process, which is executed by the executionunit includes a process of notifying information relating to the resultof the detection of the marker by the detection unit.

11. An information processing method executed by an informationprocessing apparatus, the method including detecting a difference from apreviously detected marker in a predetermined marker, by imagerecognition processing on an image obtained by image-capturing anarrangement place of an article; and executing a process in accordancewith the difference.

12. The information processing method executed by an informationprocessing apparatus according to 11, the method further includingstoring a result of the detection of the detected marker in a storageapparatus in a time-series manner and obtaining a difference between hepreviously detected marker and the currently detected marker, in which aplurality of the markers are set on the place.

13. The information processing method executed by an informationprocessing apparatus according to 11 or 12, the method further includingidentifying a type or a place of the detected marker, and executing aprocess in accordance with the identified type or the identified placeof the marker, in which a plurality of the markers are set to on theplace.

14. The information processing method executed by an informationprocessing apparatus according to 13, the method further includingexecuting a process in accordance with a combination of the detectedmarkers, in which a plurality of the markers are set t on the place.

15. The information processing method executed by an informationprocessing apparatus according to 13 or 14, the method further includingidentifying a detected range of the marker; and executing a process inaccordance with the identified range.

16. The information processing method executed by an informationprocessing apparatus according to any one of 11 to 15, the methodfurther including detecting a marker using a feature of the marker thatis learnt by machine learning.

17. The information processing method executed by an informationprocessing apparatus according to any one of 11 to 16, in which thepredetermined process, which is executed, includes a process oftransmitting that transmits predetermined information to a predeterminedterminal.

18. The information processing method executed by an informationprocessing apparatus according to 17, in which the predeterminedinformation includes information that identifies an image obtained byimage-capturing by an image capture unit that corresponds to thedetected marker.

19. The information processing method executed by an informationprocessing apparatus according to 17 or 18, in which the predeterminedinformation includes alert information.

20. The information processing method executed by an informationprocessing apparatus according to any one of 11 to 19, in which thepredetermined process, which is executed, includes a process ofnotifying information relating to the result of the detection of themarker.

21. A program causing a computer to execute: a procedure for detecting adifference from a previously detected marker in a predetermined marker,by image recognition processing on an image obtained by image-capturingan arrangement place of an article; and a procedure for executing aprocess in accordance with the difference.

22. The program according to 21, causing the computer to further executea procedure for storing a result of the detection of the detected markerin a storage apparatus in a time-series manner and a procedure forobtaining a difference between the previously detect maker and thecurrently detected marker, in which a plurality of the markers are seton the place.

23. The program according to 21 or 22, causing the computer to furtherexecute a procedure for identifying a type or a place of the detectedmarker, and a procedure for executing a process in accordance with theidentified type or the identified place of the marker, in which aplurality of the markers are set to on the place.

24. The program according to 23, causing the computer to further executea procedure for executing a process in accordance with a combination ofthe detected markers, in which a plurality of the markers are set on theplace.

25. The program according to 23 or 24, causing the computer to furtherexecute a procedure for identifying a detected range of the marker, anda procedure for executing a process in accordance with the range that isidentified in the procedure for identification.

26. The program according to any one of 21 to 25, causing the computerto further execute a procedure for detecting a marker using a feature ofthe marker that is learnt by machine learning.

27. The program according to any one of 21 to 26, in which thepredetermined process, which is executed in the procedure for executionincludes a process of transmitting predetermined information to apredetermined terminal.

28. The program according to 27, in which the predetermined informationincludes information that identifies an image obtained byimage-capturing by an image capture unit that corresponds to thedetected marker.

29. The program according to 27 or 28, in which the predeterminedinformation includes alert information.

30. The program according to any one of 21 to 29, in which thepredetermined process that is determined in advance, which is executedin the procedure for execution, includes a process of notifyinginformation relating to the result of the detection of the marker in theprocedure for detection.

This application claims priority based on Japanese Patent ApplicationNo. 2016-056565, filed on Mar. 22, 2016, the contents of which isincorporated by reference herein in its entirety.

1. An information processing apparatus comprising: a detection unit thatdetects a difference from a previously detected state in a predeterminedmarker, by image recognition processing on an image obtained byimage-capturing an arrangement place of an article; and an executionunit that executes a predetermined process in accordance with a timenecessary for a change in a predetermined value in the difference. 2.(canceled)
 3. The information processing apparatus according to claim 1,further comprising: an identification unit that identifies a type or aplace of the detected marker, wherein a plurality of the markers are seton the place, and wherein the execution unit executes a process inaccordance with the identified type or the identified place of themarker.
 4. The information processing apparatus according to claim 3,wherein a plurality of the markers are set on the place, and wherein theexecution unit executes a process in accordance with a combination ofthe detected markers.
 5. (canceled)
 6. The information processingapparatus according to claim 1, wherein the detection unit detects amarker using a feature of the marker that is learnt by machine learning.7-9. (canceled)
 10. The information processing apparatus according toclaim 1, wherein the predetermined process, which is executed by theexecution unit includes a process of notifying information relating tothe result of the detection of the marker by the detection unit.
 11. Aninformation processing method executed by an information processingapparatus, the method comprising: detecting a difference from apreviously detected state in a predetermined a marker, by imagerecognition processing on an image obtained by image-capturing aninstallation place of an article; and executing a process in accordancewith a time necessary for a change in a predetermined value in thedifference. 12-20. (canceled)
 21. A non-transitory computer readablestorage medium having a program causing a computer to execute: aprocedure for detecting a difference from a previously detected state ina predetermined marker, by image recognition processing on an imageobtained by image-capturing an arrangement place of an article; and aprocedure for executing a process in accordance with a time necessaryfor a change in a predetermined value in the difference. 22-30.(canceled)
 31. The information processing apparatus according to claim1, wherein the execution unit executes the predetermined process whenthe time necessary for the change in the difference equal to or morethan the predetermined value is lower than a threshold value.
 32. Theinformation processing apparatus according to claim 1, wherein themarker includes a plurality of numeral symbols, the plurality of numeralsymbols respectively corresponds to ranges of the arrangement place ofthe article, and the detection unit detects, as the difference, adifference between value of numeral previously recognized from themarker and value of numeral currently recognized from the marker. 33.The information processing apparatus according to claim 32, wherein theplurality of numeral symbols are arranged in ascending or descendingorder with respect to a direction of stacking or horizontally placingthe articles, so as to appear a new numeral symbol in ascending ordescending order as the number of the articles decreases, the detectionunit obtains, when the numeral is not recognized, a predeterminedmaximum or minimum value of the plurality of the numeral symbols as thecurrent numeral, obtains, when at least one of the numeral symbols isrecognized, a maximum or minimum value of the recognized numeral symbolas the current numeral, and detects, as the difference, a differencebetween the previously obtained numeral and the currently obtainednumeral.
 34. The information processing apparatus according to claim 1,wherein the execution unit executes, as the predetermined process, aprocess of outputting predetermined information.
 35. The informationprocessing apparatus according to claim 34, wherein the predeterminedinformation includes a message to notice of the possible occurrence of atheft.
 36. The information processing apparatus according to claim 35,wherein the predetermined information includes information thatidentifies an image obtained by image-capturing by an image capture unitthat corresponds to the detected marker.
 37. The information processingapparatus according to claim 35, wherein the predetermined informationincludes alert information.